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PREFACE 


This  report  describes  the  results  of  a  field  experiment 
conducted  to  explore  the  effects  of  two  variables  on  achievement 
in  computer-based  training.  The  two  variables  were  the  presence 
or  absence  of  interaction  between  student  and  instructor,  and 
paired  versus  individual  study. 

This  research  was  conducted  under  the  United  States  Air 
Force  Summer  Faculty /Graduate  Student  Research  Program.  The 
research  was  sponsored  by  the  Air  Force  Office  of  Scientific 
Research/ AFSC,  United  States  Air  Force,  under  contract  F49620-90- 
0076. 


SUMMARY 


Research  suggests  that  small  group  study  and  interactions 
between  student  and  teacher  can  improve  performance  in  computer- 
based  training  (CBT) .  However,  there  is  no  available  research  on 
the  interaction  between  these  two  variables.  Therefore,  a  2  x  2 
factorial  design  experiment  was  conducted.  The  goals  were  to 
examine  the  effects  on  achievement  of  student-instructor 
interactions,  paired/ individual  study,  and  main  effect 
interactions.  Two  hypotheses  were  tested.  The  first  hypothesis 
was  that  paired  learning  would  have  a  positive  effect  on 
achievement.  The  second  was  that  interactions  between  student 
and  instructor  would  have  a  positive  effect  on  achievement.  The 
experimental  results  support  the  first  hypothesis.  They  provide 
only  moderate  support  for  the  second  hypothesis.  Perhaps  a  team 
partner  provides  the  feedback  and  support  usually  provided  by  the 
instructor  in  a  traditional  classroom  setting. 

Individual  study  may  not  be  the  best  arrangement  for  CBT. 
Higher  achievement  is  found  when  students  work  CBT  in  pairs.  The 
approach  to  CBT  development  should  include  social  factors  during 
planning  and  implementation.  More  research  is  needed  on  the 
social  aspects  of  learning  and  the  role  of  the  instructor  in  CBT. 
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THE  EFFECTS  OF  STUDENT-INSTRUCTOR  INTERACTION 
AND  PAIRED/ INDIVIDUAL  STUDY  ON  ACHIEVEMENT 
IN  COMPUTER-BASED  TRAINING  (CBT) 


I.  INTRODUCTION 

The  effects  of  environmental  factors  on  achievement  in 
computer-based  training  (CBT)  have  not  been  extensively  studied. 
However,  McCombs  et  al.  (1984)  did  find  that  two  such  factors 
were  critical  to  the  success  of  CBT  courses.  They  were:  (a) 
adequate  opportunities  for  student- instructor  interactions,  and 
(b)  the  addition  of  group  activities  within  individualized 
training. 

Frequent  but  short  interactions  between  the  student  and 
instructor  is  consistently  reported  as  a  positive  instructor 
behavior  for  traditional  instruction  (TI) .  An  increase  in 
student- instructor  interactions  usually  produces  an  increase  in 
achievement  (Brophy,  1986;  Brophy  &  Good,  1986;  Rosenshine, 

1983).  Stephenson  (1991)  examined  the  effect  of  such 
interactions  in  CBT.  He  found  interaction  between  student  and 
instructor  had  a  positive  effect  on  achievement.  This  effect 
held  even  when  the  interactions  were  unrelated  to  the  CBT 
content . 

CBT  is  typically  conducted  in  a  one  student-one  terminal 
setting.  For  that  reason  the  second  McCombs  factor,  group 
activities,  is  seldom  found.  However,  group  activities  in  CBT 
can  occur  in  other  ways.  For  example,  students  can  work  CBT  in 
pairs.  There  are  two  bases  for  using  such  group  activity  in  CBT. 

First,  there  is  a  body  of  TI  literature  comparing  students 
working  in  groups  to  students  working  individually.  This  work 
has  focused  on  the  effect  of  cooperation  versus  competition.  The 
consensus  is  that  students  working  in  small  groups  perform  better 
than  students  working  alone.  This  is  especially  true  in  a 
cooperative  setting  (Johnson  et  al.,  1985;  Warring  et  al.,  1985; 
Yager  et  al.,  1985).  The  best  group  size  is  either  two  or  three 
(Cox  &  Berger,  1985;  Trowbridge  &  Durnin,  1984;  Webb,  1987). 
Studies  have  also  found  that  paired  students  should  be  of  the 
same  sex  and  have  similar  abilities  (Dalton,  1990;  Dossett  & 
Hulvershorn,  1983;  Hooper  et  al.,  1989;  Johnson  et  al.,  1985). 

Recent  CBT  research  provides  the  second  basis  for  arranging 
CBT  students  in  groups.  Achievement  of  students  working  CBT  in 
groups  of  two  or  three  equals  or  surpasses  achievement  of 
students  working  CBT  alone  (Carrier  &  Sales,  1987;  Cox  &  Berger, 
1985;  Dalton,  1990;  Dalton  et  al.,  1989;  Dossett  &  Hulvershorn, 
1983;  Hmelo,  1989;  Johnson  et  al.,  1986;  Justen  et  al.,  1990; 
Shull,  1990;  Trowbridge  &  Durnin,  1984;  Webb,  1987).  "No  study 
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has  reported  significantly  greater  learning  when  students  work 
alone  (Webb,  1987,  p.  195)." 

Both  student- instructor  interaction  and  paired  learning  can 
increase  achievement  in  CBT.  However,  there  is  no  available 
research  on  the  interaction  between  these  two  variables. 
Therefore,  a  2  x  2  factorial  design  experiment  was  conducted. 
The  goal  was  to  explore  the  effects  on  achievement  of  student- 
instructor  interaction,  paired/ individual  learning,  and  main 
effect  interactions.  Two  hypotheses  were  tested.  First, 
student- instructor  interaction  would  have  a  positive  effect  on 
achievement.  Second,  paired  learning  would  have  a  positive 
effect  on  achievement. 


II.  METHOD 


Subjects  fSs) 

Subjects  were  84  business  statistics  students.  For  a  class 
project  they  used  a  computer  spreadsheet  to  calculate  statistics. 
All  Ss  completed  a  pre-survey  to  assess  their  personal  computer 
(PC)  and  spreadsheet  experience. 


Experimental  Materials 

The  CBT  software  consisted  of  a  spreadsheet  tutorial  which 
was  part  of  a  larger  commercial  software  package.  The  software 
package  had  been  designed  for  an  integrated  spreadsheet-word 
processing-database  program.  The  tutorial  was  basically  linear 
and  learner-controlled.  Ss  could  repeat  a  lesson  if  desired. 

For  this  study,  the  larger  tutorial  was  altered  to  include 
only  the  introduction  plus  that  part  of  the  package  devoted  to 
using  the  spreadsheet.  The  introduction  portion  (Part  A) 
contained  four  lessons,  and  the  spreadsheet  portion  (Part  B) 
contained  eight.  The  tutorials  were  run  on  Tandy  1000SX  PCs. 

An  exercise  to  test  mastery  of  the  spreadsheet  commands  was 
added  to  the  software.  Since  the  subjects  were  business 
statistics  students,  the  exercise  used  statistical  calculations 
as  the  vehicle  for  testing  spreadsheet  mastery.  The  experimental 
material,  then,  consisted  of  a  CBT  spreadsheet  tutorial  with  a 
statistics-based  exercise.  Ss  worked  the  exercise  on  the 
computer . 
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Procedure 

Ss  were  randomly  assigned  by  spreadsheet/PC  experience  to  one 
of  four  groups.  Group  I  (n=20)  worked  in  pairs  and  received 
instructor-initiated  interactions.  Group  II  (n=25)  also  worked 
in  pairs  but  did  not  interact  with  the  instructor.  Group  III 
(n=20)  worked  individually  with  student-instructor  interactions. 
Group  IV  (n=19)  worked  individually  without  instructor 
interaction.  Ss  were  assigned  to  teams  based  on  grade  point 
average,  college  major,  and  sex.  Teams  were  not  forced  into 
either  a  cooperative  or  a  competitive  mode.  The  partners  could 
interact  with  each  other  as  desired.  All  Ss  worked  the 
statistics  exercise  individually. 

All  groups  worked  the  tutorial  in  three  sessions.  In  session 
one,  all  groups  started  on  lesson  1A  and  worked  for  70  minutes. 

In  the  second  session,  all  groups  started  on  lesson  B1  and  worked 
for  70  minutes.  In  the  third  session,  all  groups  started  on 
lesson  B3  and  worked  for  40  minutes.  This  gave  everyone  a  single 
exposure  to  lessons  A1  though  A4  and  repeated  exposure  to  lessons 
B1  through  possibly  B8,  the  spreadsheet.  Since  Ss  worked  at 
their  own  speed,  total  individual  subject  time  on  task  varied. 

After  40  minutes  on  day  3,  all  Ss  were  given  30  minutes  to 
complete  the  statistics  exercise.  Ss  worked  the  exercise 
individually. 

At  the  beginning  of  day  1  the  instructor  interacted  with  all 
teams/ individuals  to  insure  that  they  were  properly  logged  into 
the  tutorial.  Later,  the  instructor  responded  to  all  student- 
initiated  interactions  with  one  or  more  of  three  responses. 

These  were:  (1)  "Try  pushing  the  [ESCAPE]  key;"  (2)  "Try  pushing 
the  [SPACE]  bar;"  or  (3)  "Re-boot  the  system  and  start  over." 

The  suggestions  were  given  in  sequence.  If  "Try  pushing  the 
[ESCAPE]  key,"  did  not  correct  the  problem,  then  the  S  was  told 
to  "Try  pushing  the  [SPACE]  bar."  For  the  groups  not  receiving 
student- instructor  interactions,  these  suggestions  were  the  only 
instructor  interactions  experienced  after  the  startup  on  day  1. 

Two  groups  of  Ss  also  received  instructor-initiated 
interactions.  In  the  first  session,  the  instructor  initiated 
four  interactions  with  each  team/ subject.  In  sessions  two  and 
three  the  instructor  initiated  three  and  one  interactions 
respectively.  These  interactions  were  related  to  location  of 
keys  on  the  Tandy  keyboard.  For  example,  shortly  before  the  Back 
Slash  (\)  key  was  needed  in  the  tutorial  the  instructor  would 
tell  the  students  where  that  key  was  located.  Key  location  was 
explained  and  diagrammed  in  instructions  given  to  all.  For  most 
Ss,  however,  key  location  on  the  Tandy  keyboard  was  a  minor 
problem.  This  was  due  to  previous  exposure  to  an  IBM  keyboard. 
Instructor-initiated  interactions  lasted  5-10  seconds. 
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The  instructor  provided  only  information  which  was  already 
available  in  the  instructional  materials.  The  instructor 
refrained  from  commenting,  providing  feedback,  or  praising 
performance. 


Dependent  Measures 

Two  dependent  measures  were  recorded.  First,  performance  on 
the  statistics  exercise  was  scored.  Second,  Ss  recorded  the 
spreadsheet  commands  they  used  to  complete  the  exercise.  Most 
spreadsheet  procedures  can  be  performed  in  more  than  one  way.  A 
cell  entry,  for  example,  can  be  changed  with  an  EDIT  command  or 
by  re-typing  the  entry.  This  second  measure  was  recorded  to 
assess  how  many  different  spreadsheet  commands  were  actually  used 
during  the  exercise. 


III.  RESULTS 

Tables  1  and  2  show  means  and  standard  deviations  for 
Spreadsheet  Performance  and  Use  of  Spreadsheet  Commands.  ^ables 
3  and  4  show  the  analysis  of  variance. 


Table  1 

SPREADSHEET  PERFORMANCE 
Means  and  Standard  Deviations 

SETTING  INTERACTION 


Yes 

No 

Paired 

Low 

Experience 

44.50/18.17 

(n=10) 

42.86/14.10 

(n=14) 

High 

Experience 

77.00/12.52 

(n=10) 

71.36/16.45 

(n=ll) 

Individual 

Low 

Experience 

42.73/14.55 

(n=ll) 

31.67/13.92 

(n=9) 

High 

Experience 

63.89/14.09 

(n=9) 

63.50/11.56 

(n=10) 
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Table  2 


USE  OF  SPREADSHEET  COMMANDS 
Means  and  Standard  Deviations 


SETTING 


INTERACTION 


Yes 


No 


Paired 


Individual 


Low  Experience 
High  Experience 
Low  Experience 
High  Experience 


17.00/6.75 

(n=10) 

29.44/4.52 

(n=10) 

16.36/4.52 

(n=ll) 

21.11/6.51 

(n=9) 


18.57/6.91 

(n=14) 

23.64/9.51 

(n=ll) 

19.44/5.83 

(n=9) 

24.50/6.85 

(n=10) 


Table  3 

SPREADSHEET  PERFORMANCE 
Analysis  of  Variance 


Source 

SS 

DF 

Dyad 

1389.61 

1 

Interaction 

507.59 

1 

Experience 

17087.60 

1 

Dyad  x  Inter 

35.97 

1 

Dyad  x  Exper 

61.40 

1 

Intr  x  Exper 

39.97 

1 

Dyad  x  Inter 
x  Exper 

236.91 

1 

Error 

15943.44 

76 

MS 

F-Ratio 

P-Value 

1389.61 

6.624 

0.012 

507.59 

2.420 

0.124 

17087.60 

81.454 

0.000 

35.97 

0.171 

0.680 

61.40 

0.293 

0.590 

39.97 

0.191 

0.664 

236.91 

1.129 

0.291 

209.78 
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Table  4 


USE  OF  SPREADSHEET  COMMANDS 
Analysis  of  Variance 


Source 

SS 

DF 

MS 

F-Ratio 

P-Value 

Dyad 

34.57 

1 

34.57 

0.745 

0.391 

Interaction 

23.72 

1 

23.72 

0.511 

0.477 

Experience 

822 . 61 

1 

822.61 

17.728 

0.000 

Dyad  x  Inter 

152.45 

1 

152.45 

3 .285 

0.074 

Dyad  x  Exper 

79.93 

1 

79.93 

1.723 

0.193 

Intr  x  Exper 

67.56 

1 

67.56 

1.456 

0.231 

Dyad  x  Inter 
x  Exper 

40.90 

1 

40.90 

0.881 

0.351 

Error 

3526-63 

76 

46.40 

Spreadsheet  Performance 

Ss  who  worked  in  pairs  performed  better  than  Ss  who  worked 
individually.  Ss  who  interacted  with  the  instructor  outperformed 
Ss  who  did  not  interact  with  the  instructor.  However,  Table  1 
shows  that  the  No  Interaction/Low  Experience  Ss  had  the  lowest 
average  score  of  all  groups.  This  group  pulled  down  the  overall 
average  score  of  the  No  Interaction  Ss.  In  two  of  the  four 
comparisons  the  No  Interaction  Ss  performed  as  well  as  the 
Interaction  Ss.  As  expected,  High  Experience  Ss  performed  better 
than  Low  Experience  Ss. 


Use  of  Spreadsheet  Commands 

High  Experience  Ss  used  more  commands  than  did  Low 
Experience  Ss.  There  was  also  a  significant  (p<.074)  interaction 
between  the  learning  setting  and  Instructor  Interaction.  The 
means  shown  in  Table  2  suggest  that  this  was  due  to  the  No 
Interaction  Ss.  No  Interactions  Ss  in  the  Individual  setting 
used  more  commands  than  did  their  interaction/ individual 
counterparts.  In  the  Paired  setting,  however,  Interaction  Ss  on 
average  used  more  commands. 
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IV.  DISCUSSION 


Stephenson  (1991)  had  found  that  instructor  interaction  had  a 
positive  effect  on  achievement  in  CBT.  In  that  study  all  Ss 
worked  individually.  In  the  present  study  instructor  interaction 
had  little  or  no  effect  when  Ss  worked  in  pairs.  Perhaps  the 
social  functions  usually  performed  by  an  instructor  are  performed 
by  the  CBT  team  partner.  Instructor  interaction  did  not  have  an 
excessively  larger  effect  on  those  paired  Ss  without  prior 
experience,  a  result  reported  by  Stephenson  (1991) .  In  this 
study  low  experience  Ss  working  in  pairs  without  instructor 
interaction  performed  as  well  as  low  experience  Ss  working  in 
pairs  with  instructor  interaction.  This  suggests  that  the  team 
partner  can  provide  the  feedback,  support,  and  social  functions 
usually  provided  by  the  instructor  in  a  traditional  classroom. 

Ss  who  worked  individually  and  received  instructor 
interaction  performed  slightly  (but  not  significantly)  lower  than 
Ss  who  worked  in  pairs.  The  instructor  may  provide  a  social 
dimension  to  learning  which  a  partner  can  provide  in  the  paired 
setting.  The  lowest  scoring  Ss  were  those  who  worked  CBT 
individually  without  instructor  interaction.  This  result  was 
also  reported  by  Stephenson  (1991).  A  lack  of  social  interaction 
impacts  weak  students  more  than  strong  students.  Conversely, 
weak  students  benefit  more  from  social  interaction  than  strong 
students. 

The  Use  of  Spreadsheet  Commands  interaction  between  setting 
and  instructor  interaction  is  hard  to  explain.  Stephenson  (1991) 
reported  that  Use  of  Commands  did  not  vary  between  Ss  receiving 
or  not  receiving  instructor  interaction.  All  Ss  learned  the 
commands  equally  well  but  some  used  the  commands  better.  To  a 
large  degree,  this  was  also  true  in  the  present  study.  The  only 
notable  result  may  be  that  High  Experience  Ss  used  more 
spreadsheet  commands  than  Low  Experience  Ss. 

Overall,  these  results  emphasize  the  social  nature  of 
learning.  For  some  students  learning  is  simply  a  social  event. 

In  the  traditional  classroom  the  instructor  may  provide  most  of 
the  social  functions.  The  computer  is  unable  to  provide  these 
functions.  So  when  Ss  do  CBT  individually,  interaction  with  a 
human  instructor  has  a  notable  effect.  However,  when  social 
functions  can  be  provided  by  a  team  partner  the  need  to  interact 
with  the  instructor  is  reduced. 

These  results  question  a  frequent  justification  for  CBT,  the 
potential  for  1:1  interaction.  It  may  be  that  a  1  student :1 
computer  setting  is  not  comparable  to  a  1  student :1  instructor 
setting.  Due  to  the  social  interaction,  a  2  students: 1  computer 
situation  may  be  closer  to  the  traditional  ideal  of  1  student:l 
human  instructor. 
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The  short-term  nature  of  the  tutorial  used  here  limits 
generalization  of  the  results.  The  results  do  suggest  that  thQ 
best  CBT  environment  may  not  be  the  one  in  which  students  work 
individually.  Higher  achievement  is  found  when  students  work  CBT 
in  pairs.  The  social  functions  of  learning  are  served  well  by 
the  study  team  partner  and,  in  fact,  may  be  served  better  than  by 
an  instructor.  CBT  instructors  should  consider  arranging  CBT  in 
a  team  configuration. 

These  results  have  several  other  implications.  First,  if 
CBT  is  to  be  worked  in  pairs,  the  software  should  be  designed  for 
pairs.  Second,  the  instructor  should  be  trained  to  work  in  a 
team  setting.  Third,  the  entire  approach  to  CBT  should  emphasize 
social  facilitation.  This  last  suggestion  is  significant  since 
the  usual  reason  for  adopting  CBT  is  the  supposed  opportunity  for 
1:1  learning. 

Overall,  it  does  appear  that  the  social  aspect  of  learning 
needs  to  be  considered  in  CBT.  If  students  are  working  CBT 
individually,  then  the  instructor  (or  course  administrator)  must 
provide  the  social  functions.  If  students  are  working  in  pairs, 
then  the  instructor's  social  role  is  reduced.  In  this  second 
case  the  instructor's  role  may  become  very  specialized.  For 
example,  the  role  could  be  to  provide  social  support  for  just  the 
weaker  students.  The  role  of  the  CBT  instructor  should  be 
considered  in  the  CBT  process,  and  further  research  on  that  role 
should  be  conducted. 
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